207 research outputs found
Joint Optimization of Power Allocation and Training Duration for Uplink Multiuser MIMO Communications
In this paper, we consider a multiuser multiple-input multiple-output
(MU-MIMO) communication system between a base station equipped with multiple
antennas and multiple mobile users each equipped with a single antenna. The
uplink scenario is considered. The uplink channels are acquired by the base
station through a training phase. Two linear processing schemes are considered,
namely maximum-ratio combining (MRC) and zero-forcing (ZF). We optimize the
training period and optimal training energy under the average and peak power
constraint so that an achievable sum rate is maximized.Comment: Submitted to WCN
A Nonconvex Splitting Method for Symmetric Nonnegative Matrix Factorization: Convergence Analysis and Optimality
Symmetric nonnegative matrix factorization (SymNMF) has important
applications in data analytics problems such as document clustering, community
detection and image segmentation. In this paper, we propose a novel nonconvex
variable splitting method for solving SymNMF. The proposed algorithm is
guaranteed to converge to the set of Karush-Kuhn-Tucker (KKT) points of the
nonconvex SymNMF problem. Furthermore, it achieves a global sublinear
convergence rate. We also show that the algorithm can be efficiently
implemented in parallel. Further, sufficient conditions are provided which
guarantee the global and local optimality of the obtained solutions. Extensive
numerical results performed on both synthetic and real data sets suggest that
the proposed algorithm converges quickly to a local minimum solution.Comment: IEEE Transactions on Signal Processing (to appear
A Single-Loop Gradient Descent and Perturbed Ascent Algorithm for Nonconvex Functional Constrained Optimization
Nonconvex constrained optimization problems can be used to model a number of
machine learning problems, such as multi-class Neyman-Pearson classification
and constrained Markov decision processes. However, such kinds of problems are
challenging because both the objective and constraints are possibly nonconvex,
so it is difficult to balance the reduction of the loss value and reduction of
constraint violation. Although there are a few methods that solve this class of
problems, all of them are double-loop or triple-loop algorithms, and they
require oracles to solve some subproblems up to certain accuracy by tuning
multiple hyperparameters at each iteration. In this paper, we propose a novel
gradient descent and perturbed ascent (GDPA) algorithm to solve a class of
smooth nonconvex inequality constrained problems. The GDPA is a primal-dual
algorithm, which only exploits the first-order information of both the
objective and constraint functions to update the primal and dual variables in
an alternating way. The key feature of the proposed algorithm is that it is a
single-loop algorithm, where only two step-sizes need to be tuned. We show that
under a mild regularity condition GDPA is able to find Karush-Kuhn-Tucker (KKT)
points of nonconvex functional constrained problems with convergence rate
guarantees. To the best of our knowledge, it is the first single-loop algorithm
that can solve the general nonconvex smooth problems with nonconvex inequality
constraints. Numerical results also showcase the superiority of GDPA compared
with the best-known algorithms (in terms of both stationarity measure and
feasibility of the obtained solutions).Comment: This work has been accepted by the Thirty-ninth International
Conference on Machine Learning. (Some typos in the ICML proceedings are
corrected in this version.
A Generalized Alternating Method for Bilevel Learning under the Polyak-{\L}ojasiewicz Condition
Bilevel optimization has recently regained interest owing to its applications
in emerging machine learning fields such as hyperparameter optimization,
meta-learning, and reinforcement learning. Recent results have shown that
simple alternating (implicit) gradient-based algorithms can match the
convergence rate of single-level gradient descent (GD) when addressing bilevel
problems with a strongly convex lower-level objective. However, it remains
unclear whether this result can be generalized to bilevel problems beyond this
basic setting. In this paper, we first introduce a stationary metric for the
considered bilevel problems, which generalizes the existing metric, for a
nonconvex lower-level objective that satisfies the Polyak-{\L}ojasiewicz (PL)
condition. We then propose a Generalized ALternating mEthod for bilevel
opTimization (GALET) tailored to BLO with convex PL LL problem and establish
that GALET achieves an -stationary point for the considered problem
within iterations, which matches the iteration
complexity of GD for single-level smooth nonconvex problems.Comment: Camera ready versio
- …